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Introduction 


Over the past 30 years, an entirely new industry of “intermediary organizations” and “support 

providers” has emerged to help schools and districts improve (Hatch, 2002; Honig, 2004; Rowan, 

2002) K-3 reading outcomes in New York City. These support providers include those that focus 

on creating new schools or turning around old ones (for example organizations such as Success 

for All and Expeditionary Learning) as well as others that work directly with students, teachers, 

or both to improve instruction and outcomes in particular subject areas like reading (such as 
Reading Recovery). Illustrating the growth of this industry, in 
2001, the Catalog of School Reform Models listed 63 different 
organizations focused on school improvement with 48 (76%) 
established after 1980 and only one in existence before 1960 
(Northwest Regional Educational Laboratory, 2001). In 2018, a 
survey of 13 state websites revealed 151 of these organizations 
focused primarily on “turning around” low-performing schools 
Meyers & VanGronigen (2018). 


This brief provides a summary of the results of a project designed to help build the collective 
capacity and increase the impact of programs working to improve K-3 reading outcomes in 
New York City public elementary schools (the “K-3 reading improvement sector”). The first phase 
of this project identified all of the programs working in the sector in 2015 and then examined 
the extent to which a sample of programs have the goals, resources, personnel to improve K-3 
reading outcomes across New York City. In 2016-17, the second phase of this project mapped 
the relationships among a sample of sector programs, the sources they rely on to support 

their work, and the NYC schools with whom they partner. Making these relationships visible 
shows to what extent students from different backgrounds and schools get access to valuable 
information, resources, and expertise, and the extent to which programs are in a position to 
increase their collective impact through coordination and collaboration. 


[PODCAST] 
Thomas Hatch: Mapping New York City’s ‘School Improvement Industry’ 
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Determining the most appropriate next steps to improve the effectiveness of support providers 
depends on developing a much better sense of the number, goals, strategies, and connections 
among those providers currently at work in a given location. To fill this gap, with the support 

of the New York Community Trust and the Brooke Astor Fund for New York City Education, we 
launched a three-phase project to assess the collective capacity of the programs working in 
New York City public elementary schools to improve K-3 reading outcomes. We assessed the 
collective capacity of this K-3 reading improvement sector by addressing four questions: 


1. How many programs focus on improving reading outcomes at the K-3 level in 
New York City public elementary schools? 


Over 100 programs are working in the K-3 Reading Improvement Sector in NYC 


m Only those programs that worked directly with teachers and/or students on an ongoing 
basis in at least one New York City elementary school in 2015 were counted. (Programs 
that only provide curricula and materials or provide services for students or teachers but 
do not have an ongoing relationship with at least one school are not included.) 


m Programs included a variety of local/community-based programs, tutoring services, and 
university affiliates. 52% of the programs provide professional development to teachers; 
40% provide services directly to students; and 8%work with both teachers and students. 


2. To what extent do programs have goals, resources, and personnel that can 
contribute to improved K-3 reading outcomes? 


The programs in the reading improvement sector focus on a wide-range of reading-related 
goals, but there is no common focus and limited evidence of effectiveness 


gm Almost half of the sample programs (41%) focused on a specific literacy skill, with the 
largest number concentrating on comprehension (19%). The remaining programs 
addressed goals like getting students to read at grade level or enabling students to 
meet reading standards (see Figure 1). 


m The programs working with students were more likely to focus on specific skills or general 
goals like reading at grade level. Programs working with teachers or working with both 
teachers and students were more likely to focus on goals like Common Core standards 
or balanced literacy. 


m 72% of the sample programs reported using assessments for students focused explicitly 
on reading and another 9% of the programs assess teachers’ instruction. However, 19% 
reported that they did not assessments for either students or teachers. 


mw 19% of the programs had publicly available evaluations carried out by independent 
evaluators; 25% produced their own internal evaluations; 34% carried out internal 
progress monitoring; and 22% reported they did not carry out an evaluation or internal 
monitoring of their outcomes. 


Figure 1. Reading Goals by Program Type (percentage) 
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Although these programs are not required to, they either hire personnel with a B.A. or more or 
provide training for their staff members. 


m Every program working with teachers or working with both teachers and students 
required staff to have either a bachelor’s degree or a graduate degree, and all but one 
of these teacher programs required staff members to have teaching experience. 


gw All of the programs working directly with students provided training for their staff 
members and 62% required training specifically related to reading. However, only 23% of 
the programs working directly with students reported no specific qualifications for staff 
and four (31%) only required a high school diploma, and only one required teaching 
experience. 


3. Which NYC public elementary schools have access to the resources and support 
of the Sector? 


26 sample programs are connected to 161 different schools across all five boroughs, 
comprising 16% of all elementary schools in NYC. 


The 161 partner schools include 28% of the elementary schools in the Bronx and 26% of 
the elementary schools in Manhattan, with high concentrations in the northern parts of the 
borough (see Figure 2). 


Figure 2. Geographic map of partner schools 


PATERSON 


Magunt Ve New Rochelle 


Meaneck 
Englewood 
Hackensack 


Little Falls 
Clifton Passaic 
Cedar Grove Fort Lee, 
Ridgefield 


Nutley Cliffside Park 


Fairview 
Bloomfield 


Vest Orange 
Belleville 
Secaucus 
Union City 
East Orange Keamy 
Bs NEWARK 
JERSEY CITY Bral P 
al e 
* 
Bayonne. ‘ i 
ELIZABETH € a ® é Hey Stre 


Linden 


Carteret 


| Schools partnered with student programs 
| School partnered with teacher programs 
|| Schools partnered with both programs 


Schools partnered with multiple programs 
(2 or > 2 programs) 


Access this report at: https://repository.upenn.edu/cpre_policybriefs/85 


The Role of External Support Providers in Improving K-3 Reading Outcomes in NYC 


Figure 3. School Partners Mean % Economic Index and % Failing to Reach Proficiency 


90 


80 
Mean % of 3rd graders failing 
70 to achieve proficiency (ELA) 


iE Mean Economic Index 


60 
50 
40 
30 
20 


Schools with Schools with Schools with Schools with 
Multiple Partners One Student Partner One Teacher Partner One Both Partner 


Overall, the programs are partnering with relatively high levels of need in terms both 
performance and poverty (as measured by the percentages of students who fail to achieve 
proficiency on New York State’s 3rd grade ELA exam and by the percentage of students with 
a high “Economic Index” based on: a) the percentages of students in temporary housing; b) 
eligible for Health Reimbursement Arrangements; and c) eligible for free lunch). Nonetheless, 
these data also suggest that schools with higher levels of need are partnering with different 
numbers and kinds of program. Schools that partner with multiple programs or with student 
programs have higher levels of need than schools that partner with teacher programs and 
programs serving both students and teachers (see Figure 3). 


4. To what extent are programs positioned to increase their collective impact in the 
future? 


To get a sense of the extent to which programs in the sector are positioned to share 
information with one another, to coordinate services, and to increase their collective impact, 
we explored three questions: 


m Howare the programs in the sector connected to one another? 
m Where do the programs get their funding? 
m What sources do they turn to for expertise? 


To address these questions we conducted interviews and a social network survey with a 
sample of sector programs in 2016-17. We found: 


Just over half of thirty sample programs (57%) describe themselves as being in collaboration 
(“working informally”) or partnership (“working together formally”) with at least one other 
sample program (see Figure 4). 


m 43% of the sample programs also report being in at least monthly contact with another 
program (monthly contact is generally considered “frequent contact” and serves as the 
standard for regular exchanges of information, resources, and expertise). 


m 33% of the sample programs are working in partnership with multiple programs. 


Most of the frequently connected programs and those in collaborations/partnerships 
work directly with students or with students and teachers. 


m The programs cited most frequently as collaborators or partners tend to participate in 
collaborative groups underwritten by local funders. 


m Although a cluster of sample programs were regularly connected, over half of the 
sample programs were not in regular contact with other sample programs, and one 
third reported no frequent contacts, collaborations, or partnerships with other sector 
programs. 
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Figure 4. Network of program relationships reported among sample programs 
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Figure 5. Network of relationships between sample programs and charitable foundations, corporations, 
government agencies, and non-profit organizations 
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21 sample programs received support from 57 different funders (see Figure 5); while another 7 
programs received funding largely through school contracts 


27 sample programs identified 75 different sources for literacy expertise with very little overlap 
(see Figure 6) 


m No source was mentioned by more than four programs 
m The most common source was the National Council for Teachers of English (NCTE) 


m Three programs did not identify any sources of literacy expertise 


18 programs named 31 sources for professional development expertise 
m 12 programs did not identify any sources of professional development expertise 


m Of the 12 programs that did not name a professional development source, five were 
teacher programs, three were student programs, and three were programs serving both 
teachers and students. 


Implications & Next Steps 


Taken together, these results show that sector programs have goals, services, and personnel 
that could help to improve K-3 reading outcomes in New York City. The fact that 26 sample 
programs work in 161 of all public elementary schools in New York City shows that the programs 
in the sector may already work in a majority of New York City public elementary schools and 
could serve as a conduit for sharing resources and expertise throughout the City. However, 
the goals of the programs vary considerably; student and teacher programs differ in terms 

of their goals and personnel, and only a few programs have had formal outside evaluations 
completed. Further, different kinds of schools may be getting access to different kinds of 
support and resources. In particular, many of the highest poverty schools partner with multiple 
programs, which provides access to more resources and personnel, but may also contribute to 
fragmentation and problems of coordination. 


In addition to the variety of goals, the differences among the student and teacher programs, 
and the differences in access among partner schools, the collective impact of the sector also 
suffers from the evidence that many programs are engaged in related but largely independent 
work. Furthermore, the programs in the sector are informed by a wide range of sources of 
funding and expertise that are themselves likely to be only loosely connected. The unconnected 
programs and the disparate sources of funding and expertise mean that most of these programs 
have to demonstrate their effectiveness while working independently of other programs. The 
fact that only three of the teacher programs reported frequent contact with other sample 
programs and that five teacher programs did not name a source of professional development 
expertise also indicates that these programs may be operating even more independently than 
the student programs within the sector. Nonetheless, there are several clusters of programs that 
are frequently connected and working in collaborations or partnerships. These clusters could 
serve as a powerful force for focus and collaboration in reading improvement across the city. 


Given these results, no single strategy will be sufficient to build the collective capacity of the 
sector moving forward. Instead, different approaches are needed to take advantage of the 
different strengths of the programs working with students and those working with teachers and 
to promote greater collective impact on reading outcomes across the City. 


1. Create an inventory and “map” that shows all the programs working in the 
sector and the schools with which they partner 


Instituting a bi-annual inventory of all the sector programs and “mapping” their partner 
schools could highlight gaps in services and identify underserved schools and neighborhoods. 
Schools and programs could also use this information to make more strategic choices about 
who to work with and to facilitate coordination when schools have multiple program partners. 
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2. Identify common needs, assessments, and tools to promote coherence across the 
sector 


The NYC Department of Education or an alliance of sector programs could establish common 
areas of focus or standards to help ensure the quality and consistency of work. Programs 
could continue to address different goals and needs but the shared understanding would 
facilitate alignment and consistency. In addition to articulating key local needs or goals 
related to elementary reading, productive developments would include standards for the 
“dosage” or extent of services support providers should offer; sharing of common reading 
assessments; identification of core practices for reading instruction; and creation of workshops 
for program personnel to support their use of common assessments and effective practices. 


3. Foster strategic alliances among programs working with students and programs 
working with teachers 


Productive collaborations among student and teacher programs could both deepen their 
work and expand their reach. “Matching” support from afterschool programs for students 

and in-school programs for teachers and/or students would be particularly effective, and the 
adoption of complementary goals and common assessments could help to magnify effects. 
Identifying the obstacles and opportunities for collaborations among a small number of 
complementary programs could also help to provide a foundation for more coordinated work 
in the future. 


4. Build broader coalitions for neighborhood impact 


Schools can benefit from the support of sector programs, but many different people and 
institutions can help to create opportunities for students to learn to read that support 
collective impact in different communities and neighborhoods. In New York City, for example, 
East New York Reads (in Brooklyn) and the Neighborhood Literacy Initiative (in South Jamaica, 
Queens), already bring together some sector programs with local community organizations 
and institutions like the Brooklyn and Queens Public Libraries. 


5. Create a local “hub” to share information and expertise 


Education leaders, policymakers, and funders need to balance efforts to promote 
coordination in the sector with the recognition that imposing narrow goals and evaluation 
measures can reduce the flexibility needed to meet the local needs of schools. Creating a 
“hub” organization and/or an annual forum for educators, community members, funders, and 
researchers, could foster that balance by facilitating connections, sharing information about 
reading research, and supporting reflection on progress and sector development. 


Conclusion 


Strategies like these begin with the recognition that investments need to be made in building 
the capacity of both external support providers and schools. They promote coordination 
and coherence in the sector while striving to avoid the excessive regulation that can reduce 
flexibility and adaptability. In the process, these strategies establish a middle way between 
producing more bureaucratic requirements and letting “1000 flowers bloom.” 
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